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1.  Introduction 


1.1  Background 

In  recent  years,  there  have  been  growing  applications  of  robotic  technologies  in  fields  such  as 
space  exploration,  search  and  rescue,  national  defense,  entertainment,  police  special  weapons  and 
tactics  operations,  health  care,  and  personal  assistance  (Chen,  Haas,  &  Barnes,  2007).  These 
robotic  systems  will  extend  the  ranges  and  capabilities  of  their  human  operators’  perception  and 
action  and  will  have  a  major  impact  on  future  combat  operations  (Oron-Gilad,  Chen,  &  Hancock, 
2005).  Future  warfare  employing  robotic  systems  may  need  to  integrate  information  from  multiple 
platforms,  potentially  from  aerial  and  ground  sources.  A  human  operator’s  perception  of  remote 
environments  often  relies  on  the  video  feeds  from  the  camera(s)  mounted  on  the  robots.  Unmanned 
aerial  vehicles  (UAVs)  generally  provide  exocentric  (perspective  from  outside  the  environment) 
views  of  the  problem  space  (i.e.,  the  battlefield)  while  the  unmanned  ground  vehicles  (UGVs) 
present  viewpoints  that  are  egocentric  (perspective  from  within  the  environment)  and  immersed  in 
the  environment.  The  ideal  view  depends  on  the  task;  overall  awareness  and  pattern  recognition  are 
optimized  by  exocentric  views  whereas  the  immediate  environment  is  often  viewed  better  egocen- 
trically.  According  to  Chen,  Durlach,  Sloan,  and  Bowens  (in  press),  robotics  operators  tend  to 
prefer  using  UAVs  instead  of  UGVs  to  conduct  reconnaissance  tasks  (i.e.,  target  detection),  which 
is  consistent  with  the  literature  that  exocentric  perspective  is  more  suitable  for  global  awareness 
performance  and  search  tasks  than  egocentric  perspective  (Wang,  2004).  However,  depending  on 
the  missions,  targets  might  need  to  be  examined  at  a  closer  range  from  the  ground  after  they  are 
detected  by  UAV  operators.  Displays  for  integrating  infonnation  from  different  frames  of  refer¬ 
ence  (FORs)  (e.g.,  exocentric  and  egocentric)  present  potential  human  performance  issues  that  need 
to  be  carefully  evaluated  (Thomas  &  Wickens,  2000).  Research  has  shown  that  integrating  infor¬ 
mation  across  egocentric  and  exocentric  views  can  be  challenging  for  the  operator  (Olmos, 
Wickens,  &  Chudy,  2000;  Thomas  &  Wickens,  2001).  Essentially,  dual  displays  with  both  FORs 
require  effective  scanning  of  the  displays  and  integration  of  information  from  two  different 
perspectives  to  form  an  accurate  assessment  of  the  situation.  Furthermore,  operators  may  be 
susceptible  to  saliency  effect  and  anchoring  heuristic/bias  (Thomas  &  Wickens,  2000).  In  other 
words,  salient  infonnation  on  one  display  may  catch  most  of  the  operator’s  attention,  and  the 
operator  may  form  an  inaccurate  judgment  because  infonnation  from  the  other  sources  is  not 
properly  attended  to  and  integrated.  In  Thomas  and  Wickens  (2000),  participants  were  found  to 
tunnel  their  attention  into  the  egocentric  view  to  the  exclusion  of  information  from  the  exocentric 
view. 

Another  potential  human  performance  issue  related  to  integrating  information  from  different  FORs 
is  navigation.  In  order  to  successfully  navigate  in  the  remote  environment,  the  robotics  operator 
needs  to  have  a  good  sense  of  orientation,  both  globally  and  locally.  Globally,  the  operator  needs 
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to  know  where  the  areas  of  interest  are  relative  to  the  location  of  the  robot;  locally,  the  operator 
needs  to  negotiate  local  turns  and  avoid  obstacles  in  order  to  navigate  to  the  robot’s  destinations. 
Navigation  with  a  traditional  (north-up)  map  can  be  challenging  at  times  because  of  the  demand 
of  mental  rotation.  Studies  comparing  human  performance,  which  use  north-up  maps  (world- 
referenced;  fixed  viewpoint)  versus  track-up  (ego-referenced;  rotating  viewpoints)  maps,  con¬ 
sistently  show  that  track-up  maps  are  better  for  local  guidance  (i.e.,  navigation)  and  north-up 
maps  are  better  for  global  awareness  (Aretz,  1991;  Casner,  2005;  Darken  &  Cevik,  1999;  Lohrenz, 
Gendron,  Edwards,  Myrick,  &  Trenchard,  2004;  Wang,  2004;  Werner,  2002).  User  interface 
design  guidelines  generally  recommend  making  both  north-up  and  track-up  maps  available  (U.S. 
Department  of  Transportation,  2006).  It  is  also  recommended  that  when  one  is  in  a  route -planning 
mode,  the  default  should  be  north-up;  during  navigation,  the  default  should  be  track-up. 

1.2  Current  Study 

The  current  study  was  designed  to  gain  a  deeper  understanding  of  UAV-guided  navigation.  In 
military  operations  where  UAVs  are  used  to  provide  aerial  views  of  the  mission  environments, 
human  perfonnance  issues  are  likely  to  occur  if  entities  on  the  ground  (e.g.,  UGV  operators, 
dismounted  Soldiers,  etc.)  need  to  rely  on  the  video  feeds  from  UAVs  to  help  them  navigate  in  the 
environment.  Depending  on  the  class  of  UAV,  several  types  of  views  can  be  provided.  The  class 
I  UAVs  such  as  the  micro-air  vehicle  (MAV)  can  be  manipulated  to  provide  an  aerial  view  that  is 
more  congruent  with  the  view  from  the  ground  entities.  For  example,  the  MAV  can  travel  ahead 
of  the  UGV  to  provide  a  near  aerial  view.  However,  for  UAVs  that  are  larger,  it  is  more  likely  that 
the  aerial  vehicles  are  controlled  remotely  and  the  video  feed  can  only  be  provided  in  a  certain 
fixed  orientation.  It  is  also  likely  that  the  UAV  will  be  traveling  (e.g.,  circling  in  the  same  area) 
and  the  perspectives  from  the  UAV  will  be  constantly  changing.  If  a  ground  entity  (Soldier  or 
Soldier  via  a  UGV)  needs  to  look  for  a  target  that  is  moving  and  needs  to  navigate  based  on  video 
feed  from  a  UAV  that  s/he  cannot  control,  we  can  anticipate  human  performance  issues  such  as 
disorientation  to  occur.  As  research  shows,  track-up  maps/displays  are  more  effective  than 
fixed/north-up  maps  for  land  navigation.  Therefore,  potential  disorientation  issues  that  are 
associated  with  navigating  with  fixed/north-up  maps  can  be  anticipated  with  such  scenarios  as 
navigating  with  a  fixed  or  (even  worse)  constantly  changing  (but  not  congruent  with  the  direction 
of  the  ground  navigation)  aerial  view.  Lighting  conditions  (e.g.,  nighttime)  could  also  exacerbate 
the  disorientation  problem  since  operators  cannot  rely  on  the  color  cues  of  their  surroundings.  In 
the  current  study,  we  sought  to  evaluate  such  perfonnance  issues  and  in  ensuing  studies,  we  will 
propose  and  examine  potential  mitigation  strategies  (e.g.,  user  interface  design  and/or  training)  for 
such  performance  degrada-tions. 
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2.  Method 


2.1  Participants 

A  total  of  28  college  students  (10  females  and  18  males)  was  recruited  to  participate  in  the  study. 
The  ages  of  the  participants  ranged  from  18  to  34  (mean  [M]  =  23.43,  standard  deviation 
[SDJ  =  4.88).  Participants  were  compensated  $  15/hour  and  given  class  credit  for  their  partici¬ 
pation  in  the  experiment. 

2.2  Apparatus 
2.2.1  Simulators 

A  fi rst-pcrson-shooter  computer  game,  Half  Life21,  was  used  to  provide  the  simulation  for  the 
MAV  and  the  UGV  (figure  1).  The  terrain  database  of  the  McKenna  Military  Operations  on  Urban 
Terrain  (MOUT)  at  Fort  Benning,  Georgia,  was  used  for  this  experiment.  The  first-person-shooter 
perspective  of  Half  Life2  was  used  to  simulate  the  view  from  the  UGV.  Participants  used  voice 
commands  (e.g.,  forward,  backward,  turn  left,  turn  right,  scan  for  targets,  engage  target,  etc.)  to 
control  the  UGV’s  navigation.  Half  Life2  also  provides  a  spectator’s  view,  which  was  used  to 
simulate  the  view  from  the  MAV.  Participants  used  a  joystick  to  control  the  movement  of  the 
MAV. 

Another  set  of  simulations  was  used  to  provide  the  large  UAV  views.  The  large  UAV  with  fixed 
view  was  simulated  as  hovering  above  the  MOUT  at  100  meters.  The  orbiting  UAV  was  simu¬ 
lated  as  orbiting  the  MOUT  at  15  miles  per  hour  (mph)  at  the  same  altitude.  We  rendered  the 
night  vision  condition  by  adjusting  the  color  setting  of  the  computer  monitors  to  render  scenes  as 
though  seen  through  night  vision  goggles.  Participants  were  able  to  see  the  entire  MOUT  site 
from  the  larger  UAV  video.  However,  with  the  MAV,  they  could  only  fly  at  a  lower  altitude 
(roughly  the  height  of  a  three-story  building)  and  could  not  have  a  bird’s  eye  view  of  the  environ¬ 
ment  as  good  as  that  of  the  larger  UAVs.  This  constraint  was  attributable  to  the  limitation  of  the 
simulation  program  and  does  not  reflect  the  capabilities  of  the  current  MAVs  used  in  the  U.S. 
Army.  Figure  1  shows  the  UGV  view  (left)  and  the  large  UAV  view  (right).  The  MAV  view 
was  similar  to  the  UGV  view  but  with  a  higher/adjustable  altitude. 


'Half  Life2  is  a  registered  trademark  of  Valve  Software  Corporation. 
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Figure  1.  UGV  screen  (left)  and  UAV  screen  (right). 

2.2.2  Questionnaires  and  Spatial  Tests 

A  demographics  questionnaire  (appendix  A)  was  administered  at  the  beginning  of  the  training 
session.  The  Cube  Comparison  and  the  Hidden  Patterns  tests  (Ekstrom,  French,  &  Harman,  1976) 
as  well  as  the  Spatial  Orientation  Test  were  used  to  assess  participants’  spatial  ability  (SpA).  The 
Cube  Comparison  test  requires  participants  to  compare,  in  3  minutes,  2 1  pairs  of  six-sided  cubes 
and  determine  if  the  rotated  cubes  are  the  same  or  different.  The  Hidden  Patterns  test  measures 
flexibility  of  closure  and  involves  identifying  specific  patterns  or  shapes  embedded  within  dis¬ 
tracting  information.  The  Orientation  test,  modeled  after  the  cardinal  direction  test  developed  by 
Gugerty  and  his  colleagues  (Gugerty  &  Brooks,  2004)  is  a  computerized  test  consisting  of  a  brief 
training  segment  and  32  test  questions.  Both  accuracy  and  response  time  were  automatically 
captured  by  the  program.  A  map-reading  test  (Money  &  Alexander,  1966)  was  converted  into  a 
computerized  test  via  the  software  program  E-Prime2  so  that  both  speed  and  accuracy  could  be 
captured.  A  survey  on  perceived  sense  of  direction  (Hegarty,  Richardson,  Montello,  Lovelace,  & 
Subbiah,  2002),  Santa  Barbara  Sense  of  Direction  Scale  (SBSOD)  (appendix  B),  was  also  used  to 
assess  participants’  perceived  abilities  on  navigation  and  way-finding  tasks.  Hegarty  et  al.  (2002) 
reported  that  this  self-reported  sense  of  direction  is  correlated  with  some  spatial  task  performance 
(e.g.,  imagining  oneself  taking  a  different  perspective  in  the  environment  and  learning  the  spatial 
layout  of  the  environment). 

Participants’  perceived  workload  was  evaluated  with  the  computer-based  version  of  National 
Aeronautics  and  Space  Administration  task  load  index  (NASA  TLX)  questionnaire  (appendix  C, 
Hart  &  Staveland,  1988).  According  to  Noyes  and  Bruneau  (2007),  computer-based  NASA  TLX 
tends  to  generate  higher  workload  ratings  compared  with  the  traditional  paper-based  survey. 
However,  since  the  ratings  were  used  to  compare  the  workload  levels  across  the  experimental 
conditions,  the  elevated  ratings  should  not  affect  these  comparisons. 


7  .  . 

“E-Prime  is  a  registered  trademark  of  Psychology  Software  Tools,  Inc. 
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2.3  Experimental  Design 


The  overall  design  of  the  study  was  a  2  x  2  x  4  mixed  design.  The  between-subject  variable  was 
Lighting  (day  versus  night  vision).  The  within-subject  variables  were  Target  (stationary  versus 
moving  target)  and  UAV  (no  UAV  versus  MAV  versus  Large  UAV  Fixed  View  versus  Large 
UAV  Orbiting  View). 

2.4  Procedure 

After  being  briefed  about  the  purpose  of  the  study  and  signing  the  infonned  consent  form, 
participants  completed  the  demographic  questionnaire,  followed  by  the  SpA  tests  and  the  SBSOD 
survey.  Participants  then  received  training  by  going  through  a  PowerPoint3  based  tutorial  and 
practice  on  the  tasks  they  would  need  to  conduct.  The  participants  then  completed  practice 
scenarios  using  the  different  types  of  UAVs.  They  practiced  detecting  the  targets  (both  primary 
and  secondary)  using  the  UAVs  and  then  navigating  the  UGV  to  the  locations  of  the  targets. 

They  also  practiced  placing  the  targets  on  a  map  after  the  UGV  engaged  the  targets.  The  training 
session  lasted  about  1  hour. 

Participants  then  were  randomly  assigned  to  the  day  or  night  vision  group.  In  the  experimental 
session,  the  participants  were  asked  to  look  for  the  primary  target  (an  enemy  vehicle,  which  was  a 
sport  utility  vehicle  [SUV])  by  using  the  UAV  first,  and  then  they  tele-operated  their  UGV  (i.e., 
navigated  by  voice  commands)  to  the  location  of  the  target  to  engage  it.  The  voice  commands 
were  then  executed  by  one  of  the  experimenters.  The  video  from  the  UAV  was  available  when  the 
participant  navigated  in  the  environment  using  the  UGV.  In  the  case  of  the  baseline  condition,  the 
participant  only  used  his/her  UGV  to  locate  the  primary  target.  In  the  case  of  moving  targets,  the 
participants  needed  to  ensure  continuous  monitoring  of  the  targets.  For  example,  they  needed  to 
control  the  MAV  so  that  it  followed  the  movement  of  the  target.  The  large  UAV  could  not  be 
manipulated  but  the  view  covered  the  entire  MOUT  environment.  Participants  could  request  a 
change  of  view,  in  the  case  of  the  fixed  view  UAV,  when  targets  were  occluded  by  buildings. 

Only  two  orthogonal  views  were  available.  Participants  could  request  a  view  change  as  many 
times  as  necessary.  As  described  earlier,  the  fixed  view  UAV  was  simulated  as  hovering  above 
the  MOUT  at  100  meters.  The  orbiting  UAV  was  simulated  as  orbiting  the  MOUT  at  15  mph  at 
the  same  altitude. 

Participants  were  instructed  to  find  and  navigate  to  the  primary  target  (i.e.,  SUV)  first,  before  the 
five  secondary  targets  (i.e.,  stationary  enemy  soldiers).  There  were  also  friendly  civilians  in  the 
simulated  environment  to  increase  the  visual  noise  for  the  target  detection  tasks.  Participants 
marked  the  locations  of  the  targets  on  a  blank  map  after  the  UGV  engaged  the  targets.  Partici¬ 
pants  were  instructed  to  do  this  without  studying  the  video  image  of  the  UAV  screen. 


•5 

PowerPoint  is  a  registered  trademark  of  Microsoft  Corporation. 
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There  were  eight  scenarios  corresponding  to  the  2  (Target)  x  4  (UAV)  experimental  conditions. 
The  order  of  presentation  for  the  experimental  conditions  was  detennined  by  a  Williams  design 
of  Latin  square  (Phillips,  2005).  There  were  2-minute  breaks  between  scenarios.  Participants 
assessed  their  perceived  workload  (NASA  TLX)  after  each  scenario.  After  the  eight  scenarios, 
participants  were  administered  a  Landmark  Location  test.  They  were  shown  pictures  of  five 
buildings  in  the  MOUT  environment  and  were  asked  to  mark  the  locations  of  these  buildings 
on  a  blank  map.  The  experimental  session  lasted  about  1.5  hours. 

2.5  Measures 

The  dependent  measures  included  mission  perfonnance  (i.e.,  the  number  of  targets  detected  with 
the  robotic  assets  and  the  amount  of  time  it  took  the  participants  to  find  the  targets)  as  well  as 
participants’  perceived  workload. 


3.  Results 


3.1  Operator  Performance 

3.1.1  Target  Search  Time 

Participants  were  designated  as  high  SpA  or  low  SpA,  based  on  their  composite  SpA  test  scores 
(median  split).  We  derived  the  composite  scores  by  summing  the  participants’  rank  on  each  spatial 
test.  A  mixed  analysis  of  variance  (ANOVA)  was  performed  to  examine  the  search  time  for  the 
primary  target  (SUV),  with  the  Lighting  condition  (day  versus  night  vision)  as  the  between-subject 
factor,  the  Target  type  (stationary  versus  moving  target)  and  UAV  type  (no  UAV  versus  MAV 
versus  Large  UAV  Fixed  View  versus  Large  UAV  Orbiting  View)  as  the  within-subject  factors. 
The  analysis  revealed  that  UAV  condition  significantly  affected  the  speed  of  the  search,  F( 3,  22) 

=  6.29,  p  <  .005.  Post  hoc  (least  significant  difference  [LSD])  tests  showed  that  the  NoUAV 
condition  was  significantly  slower  than  the  other  three  conditions,  and  the  MAV  condition  was 
also  significantly  slower  than  the  two  Large  UAV  conditions  (figure  2).  There  were  no  significant 
differences  between  those  with  higher  SpA  and  those  with  lower  SpA. 
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UAV-Guided  Navigation  (Search  Time-  SUV) 

Effects  of  Lighting 


UAV  Type 


Lighting 

| Day 

E3  '■■■■  Night 


Error  Bars  show  95.0%  Cl  of  Mean 
Bars  show  Means 


Figure  2.  Primary  target  search  time. 


3.1.2  Map  Marking  Accuracy 

3. 1.2.1  Target  Locations 

A  mixed  ANOVA  was  performed  to  examine  the  map  marking  accuracy  for  the  primary  target 
(SUV),  with  the  Lighting  condition  as  the  between-subject  factor,  the  Target  type  and  UAV  type 
as  the  within-subject  factors.  The  analysis  showed  that  both  UAV  and  Target  type  significantly 
affected  the  accuracy,  F{ 3,  22)  =  3.803,  p  <  .05  and  F(l,  24)  =  6.804,/?  <  .05,  respectively.  Post 
hoc  (LSD)  tests  showed  that  the  NoUAV  condition  was  significantly  worse  than  the  two  Large 
UAV  conditions.  Additionally,  MAV  was  significantly  worse  than  Fixed  view  UAV  (figure  3). 
Participants’  SpA  did  not  affect  their  performance  in  the  NoUAV  and  MAV  conditions.  However, 
in  the  two  Large  UAV  conditions,  it  made  a  significant  difference.  Participants  with  higher  SpA 
outperformed  their  counterparts  with  lower  SpA,  F{  1,  24)  =  7.193,/?  <  .05. 

We  also  evaluated  participants’  map-marking  accuracy  for  the  secondary  targets.  We  found  that 
those  with  higher  SpA  had  a  significantly  higher  accuracy  than  did  those  with  lower  SpA,  F(  1, 

24)  =  7.873,/?  <  .01  (figure  4). 
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UAV-Guided  Navigation  (Map  Marking  Accuracy-  SUV) 

Effects  of  Target  Type 


UAV  Type 


Figure  3.  Map  marking  accuracy  (SUV  only). 


Target  Type 

| — - —  Stationary 
P  .  Moving 


Error  Bars  show  95.0%  Cl  of  Mean 

Bars  show  Means 


UAV-Guided  Navigation  (Map  Marking  Accuracy-  Overall) 

Effects  of  Spatial  Ability 


Spatial  Ability 

|  ^ —  Low  Spatial 

■  High  Spatial 


Error  Bars  show  95.0%  Cl  of  Mean 

Bars  show  Means 


Figure  4.  Map  marking  accuracy  (secondary  targets). 
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3. 1.2.2  Landmark  Locations 

Of  all  the  spatial  tests  and  surveys,  it  was  found  that  participants’  Spatial  Orientation  Test  (SOT) 
score  best  predicted  their  Landmark  Location  test  scores  (i.e.,  number  of  landmarks  correctly 
marked  on  the  map),  r  =  .478,  p  =  .006.  To  further  test  the  accuracy  of  the  SOT  in  predicting  map- 
related  performance,  a  multivariate  ANOVA  was  performed  to  test  the  effects  of  SOT  score  on  the 
16  Map  Marking  Accuracy  scores  (8  on  the  primary  targets  and  8  on  the  secondary  targets)  as  well 
as  the  Landmark  Location  test.  The  analysis  revealed  that  there  was  a  significant  difference 
between  those  with  high  SOT  scores  and  those  with  low  SOT  scores,  F(  1,  19)  =  5.838,/?  <  .05. 
Participants’  self-assessed  sense  of  direction  (based  on  the  Santa  Barbara  scale)  was  also  an 
accurate  predictor  of  their  map-related  performance  (Map-Marking  Accuracy  for  the  primary  and 
secondary  targets  and  the  Landmark  Location  test  scores),  F(l,19)  =  6.515,/?  <  .05. 


3.2  Perceived  Workload 


Weighted  ratings  of  the  scales  of  the  NASA  TLX  were  used  for  this  analysis.  Participants’  self- 
assessment  of  workload  was  significantly  affected  by  UAV  condition,  F( 3,  22)  =  4.684,/?  <  .05, 
as  well  as  the  Target  type  condition,  F( 1,  24)  =  4.548,  p  <  .05  (figure  5).  Post  hoc  (LSD)  tests 
showed  that  the  participants  experienced  significantly  higher  workload  when  they  used  the  Orbit¬ 
ing  View  UAV  (M=  59.5)  than  when  they  used  the  MAV  or  the  Fixed  View  UAV.  Participants’ 
SOT  score  was  found  to  be  an  accurate  predictor  of  their  workload,  F(l,26)  =  5.121,/?  <  .05. 
Those  with  higher  SOT  scores  had  a  significantly  lower  workload  (M=  49.4)  than  did  those  with 
lower  SOT  scores  (M=  61.7). 


Perceived  Workload 

Effects  of  Target  Type 


Target  Type 

H — - —  Stationary 
□  .  Moving 


Error  Bars  show  95.0%  Cl  of  Mean 
Bars  show  Means 


Figure  5.  Perceived  workload. 


9 


4.  Discussion 


We  simulated  a  military  reconnaissance  environment  and  examined  the  perfonnance  of  ground 
robotics  operators  who  needed  to  use  sensor  images  from  a  UAV  to  navigate  their  ground  robot 
to  the  locations  of  the  targets.  We  also  evaluated  participants’  SpA  and  examined  if  it  affected 
their  performance  or  perceived  workload.  Results  showed  that  (as  expected)  participants’  target 
search  was  significantly  slower  in  the  NoUAV  condition  than  in  the  other  three  UAV  conditions. 
Additionally,  participants’  search  was  significantly  slower  with  the  MAV  than  with  the  two  large 
UAVs.  This  could  be  because  participants  could  see  the  SUV  immediately  from  the  large  UAV 
screens,  but  they  needed  to  search  the  environment  (in  a  serial  fashion)  when  using  the  MAV. 
There  did  not  appear  to  be  any  significant  differences  between  the  two  large  UAV  conditions. 

The  Lighting  conditions,  Target  type,  and  participants’  SpA  also  failed  to  affect  the  perfonnance. 

For  the  map-marking  accuracy  perfonnance,  we  found  that  both  the  UAV  and  the  Target  type 
conditions  significantly  affected  the  performance.  The  NoUAV  condition,  not  surprisingly,  was 
again  the  worst.  The  MAV  condition  appeared  to  be  the  worst  among  the  UAV  conditions,  while 
the  Fixed  View  UAV  appeared  to  support  the  best  performance.  This  performance  difference  was 
likely  attributable  to  the  limited  view  of  the  environment  from  the  MAV,  compared  with  the  larger 
UAVs.  When  the  targets  were  moving  instead  of  stationary,  participants’  marking  accuracy 
significantly  degraded,  possibly  because  of  disorientation.  Moreover,  we  observed  significantly 
superior  performance  by  those  with  higher  SpA.  The  difference  in  performance  appeared  to  be 
most  pronounced  when  the  Fixed  View  UAV  was  used.  Those  with  lower  SpA  did  not  appear  to 
take  advantage  of  the  larger  UAVs  (compared  to  the  other  two  conditions)  as  much  as  their 
counterparts  with  higher  SpA.  Additionally,  it  appears  that  the  SOT  (Gugerty  &  Brooks,  2004) 
can  accurately  predict  the  level  of  survey  knowledge  (as  indicated  by  the  Landmark  Location  test) 
of  our  participants.  It  is  interesting  to  note  that  some  previous  research  (Hurts,  2006)  did  not  find 
significant  correlations  between  spatial  abilities  and  survey  knowledge.  It  is  likely  that  the  dif¬ 
ferences  between  the  spatial  tests  used  (the  Differential  Aptitude  Test  by  Evers  &  Lucassen,  1991, 
was  used  in  Hurts,  2006)  contributed  to  these  differences  in  findings.  Indeed,  we  found  that  there 
was  a  significant  difference  in  map-related  performances  (i.e.,  Map-Marking  Accuracy  for  the 
primary  and  secondary  targets  and  the  Landmark  Location  test  scores)  between  those  with  superior 
and  poor  SOT  performance.  Additionally,  we  found  that  participants’  self  assessment  of  their 
sense  of  direction  (based  on  the  SBSOD  scale)  was  an  accurate  predictor  of  their  map-related 
performance.  Hegary  et  al.  (2002)  showed  that  the  SBSOD  scale  was  more  related  to  self-orienta¬ 
tion  with  the  environment  than  distance  estimation  and  map  drawing.  However,  our  data  did  not 
indicate  the  relationship  between  the  SBSOD  assessment  and  navigation-related  measures  (e.g., 
target  search  time).  Only  a  difference  in  map-related  performance  was  observed.  Therefore,  it 
appears  that  only  the  accuracy  of  mental  representation  of  the  environment  was  related  to 


10 


participants’  SpA  (as  measured  by  SOT)  and  sense  of  direction.  On  the  other  hand,  the  speed  of 
navigation  (at  least  as  measured  in  our  current  study)  was  not  related. 

In  tenns  of  perceived  workload,  both  the  UAV  condition  and  the  Target  type  condition  had  a 
significant  impact.  The  Orbiting  View  condition  produced  the  highest  workload  ratings.  Moving 
targets  also  induced  higher  workload,  although  they  did  not  appear  to  increase  the  workload  when 
the  UAV  was  orbiting.  Again,  the  SOT  seemed  an  accurate  predictor  of  participants’  perceived 
workload.  Those  with  higher  SOT  scores  perceived  the  tasks  as  significantly  less  taxing  as  those 
with  lower  SOT  scores. 


5.  Conclusions 


In  the  future,  it  is  expected  that  Soldiers  will  rely  heavily  on  video  from  a  UAV  to  locate  targets. 

It  is  also  expected  that  it  will  often  be  necessary  for  Soldiers  to  further  identify  the  targets  or 
engage  the  targets  after  they  are  spotted  through  the  UAV  video  by  driving  a  UGV  to  the  target 
location  or  by  navigating  to  that  location  by  themselves  (e.g.,  for  dismounted  infantry).  Either 
way,  the  ground  navigation  will  need  to  rely  on  the  video  from  the  UAV  for  guidance.  Our  results 
showed  that  operators’  overall  perfonnance  (speed  and  accuracy)  was  better  when  they  had  access 
to  images  from  larger  UAVs  with  fixed  orientations,  compared  to  other  UAV  conditions  (baseline- 
no  UAV,  MAV,  and  UAV  with  orbiting  views).  The  UAV  with  orbiting  view  was  associated  with 
the  highest  workload  ratings.  The  results  of  this  research  should  increase  our  understanding  of  the 
costs  and  benefits  of  UAV  guided  navigation. 
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Appendix  A.  Demographic  Questionnaire 


Participant# _  Age _  Major _  Date _  Gender _ 

1.  What  is  the  highest  level  of  education  you  have  had? 

Less  than  4  yrs  of  college _  Completed  4  yrs  of  college _  Other _ 

2.  When  did  you  use  computers  in  your  education?  ( Circle  all  that  apply ) 

Grade  School  Jr.  High  High  School 

Technical  School  College  Did  Not  Use 

3.  Where  do  you  currently  use  a  computer?  (Circle  all  that  apply 1 

Home  Work  Library  Other _  Do  Not  Use 

4.  For  each  of  the  following  questions,  circle  the  response  that  best  describes  you. 

How  often  do  you: 

Use  a  mouse?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  a  joystick?  Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 

Use  a  touch  screen?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  icon-based  programs/software? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 
Use  programs/software  with  pull-down  menus? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 
Use  graphics/drawing  features  in  software  packages? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 
Use  E-mail?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Operate  a  radio  controlled  vehicle  (car,  boat,  or  plane)? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 

Play  computer/video  games? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 

5.  Which  type(s)  of  computer/video  games  do  you  most  often  play  if  you  play  at  least  once  every  few  months? 

6.  Which  of  the  following  best  describes  your  expertise  with  computer?  (check  V  one) 

_ Novice 

_ Good  with  one  type  of  software  package  (such  as  word  processing  or  slides) 

_ Good  with  several  software  packages 

_ Can  program  in  one  language  and  use  several  software  packages 

_ Can  program  in  several  languages  and  use  several  software  packages 

7.  Are  you  in  your  usual  state  of  health  physically?  YES  NO 
If  NO,  please  briefly  explain: 

8.  How  many  hours  of  sleep  did  you  get  last  night? _ hours 

9.  Do  you  have  normal  color  vision?  YES  NO 

10.  Do  you  have  prior  military  service?  YES  NO  If  Yes,  how  long _ 
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Appendix  B.  Santa  Barbara  Sense  of  Direction  Scale 


Sex:  F  M  Today's  Date: _ 

Age: _ V.  2 

This  questionnaire  consists  of  several  statements  about  your  spatial  and  navigational  abilities, 
preferences,  and  experiences.  After  each  statement,  you  should  circle  a  number  to  indicate  your 
level  of  agreement  with  the  statement.  Circle  "1"  if  you  strongly  agree  that  the  statement  applies 
to  you,  "7"  if  you  strongly  disagree,  or  some  number  in  between  if  your  agreement  is 
intennediate.  Circle  "4"  if  you  neither  agree  nor  disagree. 


1 . 1  am  very  good  at  giving  directions. 

Strongly  Agree  |— |— |— | — |— |— |  Strongly  Disagree 
12  3  4  5  6  7 

2. 1  have  a  poor  memory  for  where  I  left  things. 

Strongly  Agree  |— — |— | — | — |— |  Strongly  Disagree 
1  2  3  4  5  6  7 


3. 1  am  very  good  at  judging  distances. 

Strongly  Agree  | — |— |— |— |— |— |  Strongly  Disagree 
1234567 


4.  My  "sense  of  direction"  is  very  good. 

Strongly  Agree  | — | — | — | — | — | — |  Strongly  Disagree 
1234567 

5. 1  tend  to  think  of  my  environment  in  terms  of  cardinal  directions  (N,  S,  E,  W). 

Strongly  Agree  | — | — |— |— |— |— |  Strongly  Disagree 
1234567 


6. 1  very  easily  get  lost  in  a  new  city. 

Strongly  Agree  |— | — | — | — | — | — |  Strongly  Disagree 
1  2  3  4  5  6  7 

7. 1  enjoy  reading  maps. 

Strongly  Agree  |— |— — | — |— — |  Strongly  Disagree 
1  2  3  4  5  6  7 

8. 1  have  trouble  understanding  directions. 

Strongly  Agree  |— |— — |— — |— |  Strongly  Disagree 
1234567 


9. 1  am  very  good  at  reading  maps. 

Strongly  Agree  |— |— |— | — |— |— |  Strongly  Disagree 
1  2  3  4  5  6  7 
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10. 1  don’t  remember  routes  very  well  while  riding  as  a  passenger  in  a  car. 

Strongly  Agree  |— |— |— |— |— |— |  Strongly  Disagree 
1234567 


1 1. 1  don't  enjoy  giving  directions. 

Strongly  Agree  |— |— |— |— |— |— |  Strongly  Disagree 
1234567 

12.  It’s  not  important  to  me  to  know  where  I  am. 

Strongly  Agree  | — |— — |— |— |— |  Strongly  Disagree 
1234567 

13.1  usually  let  someone  else  do  the  navigational  planning  for  long  trips. 

Strongly  Agree  | — |— |— |— |— — |  Strongly  Disagree 
1  2  3  4  5  6  7 

14. 1  can  usually  remember  a  new  route  after  I  have  traveled  it  only  once. 

Strongly  Agree  | — |— |— | — |— |— |  Strongly  Disagree 
1  2  3  4  5  6  7 

15.1  don’t  have  a  very  good  "mental  map"  of  my  environment. 

Strongly  Agree  | — | — |— |— |— |— |  Strongly  Disagree 
1  2  3  4  5  6  7 
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Appendix  C.  NASA  TLX  Questionnaire 


Please  rate  your  overall  impression  of  demands  imposed  on  you  during  the  exercise. 

1.  Mental  Demand:  How  much  mental  and  perceptual  activity  was  required  (e.g.,  thinking,  looking, 
searching,  etc.)?  Was  the  task  easy  or  demanding,  simple  or  complex,  exacting  or  forgiving? 

LOW  |— |— I— |— I— I— I— I— I— I  HIGH 
123456789  10 

2.  Physical  Demand:  How  much  physical  activity  was  required  (e.g.,  pushing,  pulling,  turning, 
controlling,  activating,  etc.)?  Was  the  task  easy  or  demanding,  slow  or  brisk,  slack  or  strenuous,  restful  or 
laborious? 


LOW  l-H-l-l-l-l-l-l-l  HIGH 
123456789  10 

3.  Temporal  Demand:  How  much  time  pressure  did  you  feel  due  to  the  rate  or  pace  at  which  the  task  or 
task  elements  occurred?  Was  the  pace  slow  and  leisurely  or  rapid  and  frantic? 

LOW  |— |— |— |— |— I— I— I— I— I  HIGH 
123456789  10 

4.  Level  of  Effort:  How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish  your  level  of 
performance? 


LOW  I— I  HIGH 

123456789  10 

5.  Level  of  Frustration:  How  insecure,  discouraged,  irritated,  stressed  and  annoyed  versus  secure, 
gratified,  content,  relaxed  and  complacent  did  you  feel  during  the  task? 

LOW  I  HIGH 

123456789  10 

6.  Performance:  How  successful  do  you  think  you  were  in  accomplishing  the  goals  of  the  task  set  by  the 
experimenter  (or  yourself)?  How  satisfied  were  you  with  your  performance  in  accomplishing  these  goals? 

LOW  |— |— |— |— |— |— I— I— |— I  HIGH 
123456789  10 
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Appendix  D.  Glossary  of  Acronyms 


ANOVA 

analysis  of  variance 

ARL 

Army  Research  Laboratory 

FCS 

Future  Combat  System 

FOR 

frame  of  reference 

LSD 

least  significant  difference 

MAV 

micro-air  vehicle 

MOUT 

military  operations  on  urban  terrain 

SBSOD 

Santa  Barbara  sense  of  direction  scale 

SOT 

spatial  orientation  test 

SpA 

spatial  ability 

SUV 

sport  utility  vehicle 

UAV 

unmanned  aerial  vehicle 

UCF 

University  of  Central  Florida 

UGV 

unmanned  ground  vehicle 
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1  DEFENSE  TECHNICAL 
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8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS  INTEGRATION 
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US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
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AWC  FIELD  ELEMENT 
ATTN  AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  348 
FT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1 172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MP  D  UNGV ARSKY 
POPE  HALL  BLDG  470  BCBL 
806  HARRISON  DR 
FT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
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1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MQ  M  R  FLETCHER 
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1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MU  M  SINGAPORE 
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BLDG  200A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MF  C  HERNANDEZ 
2421  NW  AUSTIN  RD  STE  220 
FORT  SILL  OK  73503-9042 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

1  ARMY  G1 

ATTN  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 

WASHINGTON  DC  20310-0300 

1  ARL-HRED  LIAISON 

PHYSICAL  SCIENCES  LAB 
PO  BOX  30002 

LAS  CRUCES  NM  88003-8002 


1  DIRECTOR 

UNIT  OF  ACTION  MANEUVER  BATTLE  LAB 
ATTN  ATZKUA 
BLDG  1101 
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1  DIR  FOR  PERS  TECHNOLOGIES 
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1  CODE  1142PS 
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800  N  QUINCY  STREET 
ARLINGTON  VA  22217-5000 
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UNIVERSITY  OF  MINNESOTA 
MINNEAPOLIS  MN  55455 

1  HUMAN  FACTORS  ENG  PROGRAM 
DEPT  OF  BIOMEDICAL  ENGNG 
COLLEGE  OF  ENG  &  COMPUTER  SCIENCE 
WRIGHT  STATE  UNIVERSITY 
DAYTON  OH  45435 

1  DIR  AMC-FIELD  ASSIST  IN 
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